Search Results: "ryan"

19 August 2017

Vasudev Kamath: Writing a UDP Broadcast Receiver as Python Iterator

I had to write a small Python application to listen for some broadcast message and process the message. This broadcast messages are actually sort of discovery messages to find some peers in a network. Writing a simple UDP Server to listen on a particular port was easy; but while designing an application I was wondering how can I plugin this server into my main code. There are 2 possibility
  1. Use threading module of python to send the server code in back ground and give it a callback to communicate the data to main thread.
  2. Periodically read some messages from server code and then dispose of server.
I didn't like first approach because I need to pass a callback function and I some how will end up complicating code. Second approach sounded sane but I did want to make server more like iterator. I searched around to see if some one has attempted to write something similar, but did not find anything useful (may be my Googling skills aren't good enough). Anyway so I thought what is wrong in trying?. If it works then I'll be happy that I did something different :-). The first thing for making iterator in Python is having function __iter__ and __next__ defined in your class. For Python 2 iterator protocol wanted next to be defined instead of __next__. So for portable code you can define a next function which in return calls __next__. So here is my first shot at writing BroadCastReceiver class.
from socket import socket, AF_INET, SOCK_DGRAM
class BroadCastReceiver:
    def __init__(self, port, msg_len=8192):
        self.sock = socket(AF_INET, SOCK_DGRAM)
        self.sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
        self.sock.bind(('', port))
        self.msg_len = msg_len
    def __iter__(self):
             return self
    def __next__(self):
             try:
                 addr, data = self.sock.recvfrom(self.msg_len)
                 return addr, data
             except Exception as e:
                 print("Got exception trying to recv %s" % e)
                 raise StopIteration
This version of code can be used in a for loop to read from socket UDP broadcasts. One problem will be that if no packet is received which might be due to disconnected network the loop will just block forever. So I had to modify the code slightly to add timeout parameter. So changed portion of code is below.
...
    def __init__(self, port, msg_len=8192, timeout=15):
        self.sock = socket(AF_INET, SOCK_DGRAM)
        self.sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
        self.sock.settimeout(timeout)
        self.sock.msg_len = msg_len
        self.sock.bind(('', port))
 ...
So now if network is disconnected or no packet was received for timeout period we get a socket.timeout exception due to which StopIteration will be raised causing the for loop using server as iterator to exit. This avoids us just blocking our periodic code run forever when network is disconnected or no messages are received for long time. (may be due to connected to wrong network). Now every thing looks fine but only part is if we create the server object each time our periodic code is called we will have binding issue as we did not properly close the socket once iterator has stopped. So I added socket closing code in __del__ function for the class. __del__ will be called when garbage collector try to recollect object when it goes out of scope.
...
    def __del__(self):
        self.sock.close()
So the server can be used in for loop or by passing the object of server to next built-in function. Here are 2 examples.
r = BroadCastReceiver(5000, timeout=10)
count = 0
for (address, data) in r:
    print('Got packet from %s: %s' % address, data)
    count += 1
    # do whatever you want with data
    if count > 10:
        break
Here we use an counter variable to track iteration and after some iteration we exit for loop. Another way is use for loop with range of iteration like below.
r = BroadCastReceiver(5000,  timeout=10)
for i in range(20):
    try:
        address, data = next(r)
        # do whatever you want with data
    except:
        break
Here an additional try block was needed inside the for loop to card call to next, this is to handle the timeout or other exception and exit the loop. In first case this is not needed as StopIteration is understood by for. Both use cases I described above are mostly useful when it is not critical to handle each and every packet (mostly peer discovery) and packets will always be sent. So if we miss some peers in one iteration we will still catch them in next iteration. We just need to make sure we provide big enough counter to catch most peers in each iteration. If its critical to receive each packet we can safely send this iterating logic to a separate thread which keeps receiving packets and process data as needed. For now I tried this pattern mostly with UDP protocol but I'm sure with some modification this can be used with TCP as well. I'll be happy to get feed back from Pythonistas out there on what you think of this approach. :-)
Update I got a suggestion from Ryan Nowakowski to make the server object as context manager and close the socket in __exit__ as it can't be guaranteed that __del__ will be called for objects which exists during interpreter exits. So I slightly modified the class to add __enter__ and __exit__ method like below and removed __del__
...
    def __enter__(self):
        return self
    def __exit__(self, exc_type, exc_value, traceback):
        self.sock.close()
Usage pattern is slightly modified because of this and we need to use with statement while creating object.
with BroadCastReceiver(2000) as r:
    # use server object as you wish
    ...
It is also possible to cleanly close socket without adding context manager that is adding finally statement to our try and except block in __next__. The modified code without adding context manager looks like below.
def __next__(self):
    try:
        addr, data = self.sock.recvfrom(self.msg_len)
        return addr, data
    except Exception as e:
        print("Got exception trying to recv %s" % e)
        raise StopIteration
    finally:
        self.sock.close()
When we raise StopIteration again from except block, it will be temporarily saved and finally block is executed which will now close the socket.

29 July 2017

Dirk Eddelbuettel: Updated overbought/oversold plot function

A good six years ago I blogged about plotOBOS() which charts a moving average (from one of several available variants) along with shaded standard deviation bands. That post has a bit more background on the why/how and motivation, but as a teaser here is the resulting chart of the SP500 index (with ticker ^GSCP): Example chart of overbought/oversold levels from plotOBOS() function The code uses a few standard finance packages for R (with most of them maintained by Joshua Ulrich given that Jeff Ryan, who co-wrote chunks of these, is effectively retired from public life). Among these, xts had a recent release reflecting changes which occurred during the four (!!) years since the previous release, and covering at least two GSoC projects. With that came subtle API changes: something we all generally try to avoid but which is at times the only way forward. In this case, the shading code I used (via polygon() from base R) no longer cooperated with the beefed-up functionality of plot.xts(). Luckily, Ross Bennett incorporated that same functionality into a new function addPolygon --- which even credits this same post of mine. With that, the updated code becomes
## plotOBOS -- displaying overbough/oversold as eg in Bespoke's plots
##
## Copyright (C) 2010 - 2017  Dirk Eddelbuettel
##
## This is free software: you can redistribute it and/or modify it
## under the terms of the GNU General Public License as published by
## the Free Software Foundation, either version 2 of the License, or
## (at your option) any later version.
suppressMessages(library(quantmod))     # for getSymbols(), brings in xts too
suppressMessages(library(TTR))          # for various moving averages
plotOBOS <- function(symbol, n=50, type=c("sma", "ema", "zlema"),
                     years=1, blue=TRUE, current=TRUE, title=symbol,
                     ticks=TRUE, axes=TRUE)  
    today <- Sys.Date()
    if (class(symbol) == "character")  
        X <- getSymbols(symbol, from=format(today-365*years-2*n), auto.assign=FALSE)
        x <- X[,6]                          # use Adjusted
      else if (inherits(symbol, "zoo"))  
        x <- X <- as.xts(symbol)
        current <- FALSE                # don't expand the supplied data
     
    n <- min(nrow(x)/3, 50)             # as we may not have 50 days
    sub <- ""
    if (current)  
        xx <- getQuote(symbol)
        xt <- xts(xx$Last, order.by=as.Date(xx$ Trade Time ))
        colnames(xt) <- paste(symbol, "Adjusted", sep=".")
        x <- rbind(x, xt)
        sub <- paste("Last price: ", xx$Last, " at ",
                     format(as.POSIXct(xx$ Trade Time ), "%H:%M"), sep="")
     
    type <- match.arg(type)
    xd <- switch(type,                  # compute xd as the central location via selected MA smoother
                 sma = SMA(x,n),
                 ema = EMA(x,n),
                 zlema = ZLEMA(x,n))
    xv <- runSD(x, n)                   # compute xv as the rolling volatility
    strt <- paste(format(today-365*years), "::", sep="")
    x  <- x[strt]                       # subset plotting range using xts' nice functionality
    xd <- xd[strt]
    xv <- xv[strt]
    xyd <- xy.coords(.index(xd),xd[,1]) # xy coordinates for direct plot commands
    xyv <- xy.coords(.index(xv),xv[,1])
    n <- length(xyd$x)
    xx <- xyd$x[c(1,1:n,n:1)]           # for polygon(): from first point to last and back
    if (blue)  
        blues5 <- c("#EFF3FF", "#BDD7E7", "#6BAED6", "#3182BD", "#08519C") # cf brewer.pal(5, "Blues")
        fairlylight <<- rgb(189/255, 215/255, 231/255, alpha=0.625) # aka blues5[2]
        verylight <<- rgb(239/255, 243/255, 255/255, alpha=0.625) # aka blues5[1]
        dark <<- rgb(8/255, 81/255, 156/255, alpha=0.625) # aka blues5[5]
        ## buglet in xts 0.10-0 requires the <<- here
      else  
        fairlylight <<- rgb(204/255, 204/255, 204/255, alpha=0.5)  # two suitable grays, alpha-blending at 50%
        verylight <<- rgb(242/255, 242/255, 242/255, alpha=0.5)
        dark <<- 'black'
     
    plot(x, ylim=range(range(x, xd+2*xv, xd-2*xv, na.rm=TRUE)), main=title, sub=sub, 
         major.ticks=ticks, minor.ticks=ticks, axes=axes) # basic xts plot setup
    addPolygon(xts(cbind(xyd$y+xyv$y, xyd$y+2*xyv$y), order.by=index(x)), on=1, col=fairlylight)  # upper
    addPolygon(xts(cbind(xyd$y-xyv$y, xyd$y+1*xyv$y), order.by=index(x)), on=1, col=verylight)    # center
    addPolygon(xts(cbind(xyd$y-xyv$y, xyd$y-2*xyv$y), order.by=index(x)), on=1, col=fairlylight)  # lower
    lines(xd, lwd=2, col=fairlylight)   # central smooted location
    lines(x, lwd=3, col=dark)           # actual price, thicker
 
and the main change are the three calls to addPolygon. To illustrate, we call plotOBOS("SPY", years=2) with an updated plot of the ETF representing the SP500 over the last two years: Updated example chart of overbought/oversold levels from plotOBOS() function Comments and further enhancements welcome!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

2 July 2017

Bits from Debian: New Debian Developers and Maintainers (May and June 2017)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

26 June 2017

Jonathan Dowland: Coming in from the cold

I've been using a Mac day-to-day since around 2014, initially as a refreshing break from the disappointment I felt with GNOME3, but since then a few coincidences have kept me on the platform. Something happened earlier in the year that made me start to think about a move back to Linux on the desktop. My next work hardware refresh is due in March next year, which gives me about nine months to "un-plumb" myself from the Mac ecosystem. From the top of my head, here's the things I'm going to have to address: I think that's probably it: not a big list! Notably, I'm not locked into iTunes, which I avoid where possible; Apple's Photo app (formerly iPhoto) which is a bit of a disaster; nor Time Machine, which is excellent, but I have a backup system for other things in place that I can use.

11 January 2017

Dirk Eddelbuettel: R / Finance 2017 Call for Papers

Last week, Josh sent the call for papers to the R-SIG-Finance list making everyone aware that we will have our nineth annual R/Finance conference in Chicago in May. Please see the call for paper (at the link, below, or at the website) and consider submitting a paper. We are once again very excited about our conference, thrilled about upcoming keynotes and hope that many R / Finance users will not only join us in Chicago in May 2017 -- but also submit an exciting proposal. We also overhauled the website, so please see R/Finance. It should render well and fast on devices of all sizes: phones, tablets, desktops with browsers in different resolutions. The program and registration details still correspond to last year's conference and will be updated in due course. So read on below, and see you in Chicago in May!

Call for Papers R/Finance 2017: Applied Finance with R
May 19 and 20, 2017
University of Illinois at Chicago, IL, USA The ninth annual R/Finance conference for applied finance using R will be held on May 19 and 20, 2017 in Chicago, IL, USA at the University of Illinois at Chicago. The conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure, and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management, portfolio construction, and trading. Over the past eight years, R/Finance has included attendees from around the world. It has featured presentations from prominent academics and practitioners, and we anticipate another exciting line-up for 2017. We invite you to submit complete papers in pdf format for consideration. We will also consider one-page abstracts (in txt or pdf format) although more complete papers are preferred. We welcome submissions for both full talks and abbreviated "lightning talks." Both academic and practitioner proposals related to R are encouraged. All slides will be made publicly available at conference time. Presenters are strongly encouraged to provide working R code to accompany the slides. Data sets should also be made public for the purposes of reproducibility (though we realize this may be limited due to contracts with data vendors). Preference may be given to presenters who have released R packages. Financial assistance for travel and accommodation may be available to presenters, however requests must be made at the time of submission. Assistance will be granted at the discretion of the conference committee. Please submit proposals online at http://go.uic.edu/rfinsubmit. Submissions will be reviewed and accepted on a rolling basis with a final deadline of February 28, 2017. Submitters will be notified via email by March 31, 2017 of acceptance, presentation length, and financial assistance (if requested). Additional details will be announced via the conference website as they become available. Information on previous years' presenters and their presentations are also at the conference website. We will make a separate announcement when registration opens. For the program committee:
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich

30 December 2016

Chris Lamb: My favourite books of 2016

Whilst I managed to read almost sixty books in 2016 here are ten of my favourites in no particular order. Disappointments this year include Stewart Lee's Content Provider (nothing like his stand-up), Christopher Hitchens' And Yet (his best essays are already published) and Heinlein's Stranger in a Strange Land (great exposition, bizarre conclusion). The worst book I finished, by far, was Mark Edward's Follow You Home.





https://images-eu.ssl-images-amazon.com/images/P/B010EAQLV2.01._PC__.jpg Animal QC Gary Bell, QC Subtitled My Preposterous Life, this rags-to-riches story about a working-class boy turned eminent lawyer would be highly readable as a dry and factual account but I am compelled to include it here for its extremely entertaining style of writing. Full of unsurprising quotes that take one unaware: would you really expect a now-Queen's Counsel to "heartily suggest that if you find yourself suffering from dysentery in foreign climes you do not medicate it with lobster thermidor and a bottle of Ecuadorian red?" A real good yarn.
https://images-eu.ssl-images-amazon.com/images/P/B0196HJ6OS.01._PC__.jpg So You've Been Publically Shamed Jon Ronson The author was initially recommended to me by Brad but I believe I started out with the wrong book. In fact, I even had my doubts about this one, prematurely judging from the title that it was merely cashing-in on a fairly recent internet phenomenon like his more recent shallow take on Trump and the alt-Right but in the end I read Publically Shamed thrice in quick succession. I would particularly endorse the audiobook version: Ronson's deadpan drawl suits his writing perfectly.
https://images-eu.ssl-images-amazon.com/images/P/B00IX49OS4.01._PC__.jpg The Obstacle is the Way Ryan Holiday Whilst everyone else appears to be obligated to include Ryan's recent Ego is the Enemy in their Best of 2016 lists I was actually taken by his earlier "introduction by stealth" to stoic philosophy. Certainly not your typical self-help book, this is "a manual to turn to in troubling times". Returning to this work at least three times over the year even splashing out on the audiobook at some point I feel like I learned a great deal, although it is now difficult to pinpoint exactly what. Perhaps another read in 2017 is thus in order
https://images-eu.ssl-images-amazon.com/images/P/071563335X.01._PC__.jpg Layer Cake J.J. Connolly To judge a book in comparison to the film is to do both a disservice, but reading the book of Layer Cake really underscored just how well the film played to the strengths of that medium. All of the aspects that would not have worked had been carefully excised from the screenplay, ironically leaving more rewarding "layers" for readers attempting the book. A parallel adaption here might be No Country for Old Men - I would love to read (or write) a comparative essay between these two adaptions although McCarthy's novel is certainly the superior source material.
https://images-eu.ssl-images-amazon.com/images/P/B00G1SRB6Q.01._PC__.jpg Lying Sam Harris I've absorbed a lot of Sam Harris's uvre this year in the form of his books but moreover via his compelling podcast. I'm especially fond of Waking Up on spirituality without religion and would rank that as my favourite work of his. Lying is a comparatively short read, more of a long essay in fact, where he argues that we can radically simplify our lives by merely telling the truth in situations where others invariably lie. Whilst it would take a brave soul to adopt his approach his case is superlatively well-argued and a delight to read.
https://images-eu.ssl-images-amazon.com/images/P/0140442103.01._PC__.jpg Letters from a Stoic Seneca

Great pleasure is to be found not only in keeping up an old and established friendship but also in beginning and building up a new one. Reading this in a beautifully svelte hardback, I tackled a randomly-chosen letter per day rather than attempting to read it cover-to-cover. Breaking with a life-long tradition, I even decided to highlight sections in pen so I could return to them at ease. I hope it's not too hackneyed to claim I gained a lot from "building up" a relationship with this book. Alas, it is one of those books that is too easy to recommend given that it might make one appear wise and learned, but if you find yourself in a slump, either in life or in your reading habits, it certainly has my approval.


https://images-eu.ssl-images-amazon.com/images/P/B00BHD3TIE.01._PC__.jpg Solo: A James Bond Novel William Boyd I must have read all of the canonical Fleming novels as a teenager and Solo really rewards anyone who has done so. It would certainly punish anyone expecting a Goldeneye or at least be a little too foreign to be enjoyed. Indeed, its really a pastiche of these originals, both in terms of the time period, general tone (Bond is more somber; more vulnerable) and in various obsessions of Fleming's writing, such as the overly-detailed description of the gambling and dining tables. In this universe, 007's restaurant expenses probably contributed signifcantly to the downfall of the British Empire, let alone his waistline. Bond flicking through a ornithological book at one point was a cute touch
https://images-eu.ssl-images-amazon.com/images/P/B019MMUA8S.01._PC__.jpg The Subtle Art of Not Giving A F*ck Mark Manson Certainly a wildcard to include here and not without its problems, The Subtle Art is a curious manifesto on how to approach life. Whilst Manson expouses an age-old philosophy of grounding yourself and ignoring the accumulation of flatscreen TVs, etc. he manages to do so in a fresh and provocative "21st-centry gonzo" style. Highly entertaining, at one point the author posits an alternative superhero ("Disappointment Panda") that dishes out unsolicited and uncomfortable truths to strangers before simply walking away: "You know, if you make more money, that s not going to make your kids love you," or: "What you consider friendship is really just your constant attempts to impress people." Ouch.
https://images-eu.ssl-images-amazon.com/images/P/B004ZLS5RK.01._PC__.jpg The Fourth Protocol Frederick Forsyth I have a crystal-clear memory from my childhood of watching a single scene from a film in the dead of night: Pierce Brosnan sets a nuclear device to detonate after he can get away but a double-crossing accomplice surreptitiously brings the timetable forward in order that the bomb also disposes of him Anyway, at some point whilst reading The Fourth Protocol it dawned on me that this was that book. I might thus be giving the book more credit due to this highly satisfying connection but I think it stands alone as a superlative political page-turner and is still approachable outside the machinations of the Cold War.
https://images-eu.ssl-images-amazon.com/images/P/B003IDMUSG.01._PC__.jpg The Partner John Grisham After indulging in a bit too much non-fiction and an aborted attempt at The Ministry of Fear, I turned to a few so-called lower-brow writers such as Jeffrey Archer, etc. However, it was The Partner that turned out to be a real page-turner for somewhat undefinable reasons. Alas, it appears the rest of the author's output is unfortunately in the same vein (laywers, etc.) so I am hesitant to immediately begin others but judging from various lists online I am glad I approached this one first.
https://images-eu.ssl-images-amazon.com/images/P/B00D3J2QKC.01._PC__.jpg Shogun: The First Novel of the Asian saga James Clavell Despite its length, I simply couldn't resist returning to Shogun this year although it did fatigue me to the point that I have still yet to commence on its sequel, Tai-Pan. Like any good musical composition, one is always rewarded by returning to a book and I took great delight in uncovering more symbolism throughout (such as noticing that one of the first words Blackthorne learns in Japanese is "truth") but also really savouring the tragic arcs that run throughout the novel, some beautiful phrases ("The day seemed to lose its warmth ") and its wistful themes of inevitability and karma.

22 December 2016

Shirish Agarwal: The wine and dine at debconf16

For the wine connoisseur

FOR the Wine Connoisseur

All photos courtesy KK . If any deviations, would put up labels sharing whose copyright it is. Before I get into all of that, I was curious about Canada and taking the opportunity of debconf happening there in a few months, asked few people what they thought of digital payments, fees and expenses in their country and if plastic cash is indeed used therein. The first to answer was Tyler McDonald (no idea if he is anyway related to the fast-food chain McDonalds which is a worldwide operation.) This is what he had to say/share
You can use credit / debit cards almost everywhere. Restaurant waiters also usually have wireless credit / debit terminals that they will bring to your table for you to settle your bill. How much your bank charges depends on your Canadian bank and the banking plan you are on. For instance, on my plan through the Bank Of Montreal, I get (I think) 20 free transactions a month and then after that I m charged $0.50CDN/piece. However, if I go to a Bank Of Montreal ATM and withdraw cash, there is no service fee for that. There is no service fee for using *credit* cards, only *debit* cards tend to have the fee. I live in a really rural area so I can t always get to a Bank Of Montreal machine for cash. So what I usually end up doing, is either pay by credit and then pay of the balance right away so I don t have to pay interest, or when I do use my bank card to pay for something, I ask if I can get cash back as well. Yes, Canada converted to plastic notes a few years ago. We ve also eliminated the penny. For cashless transactions, you pay the exact amount billed. If you re paying somebody in cash, it is rounded up or down to the nearest 5 cents. And for $1 or $2, instead of notes, we ve moved over to coins. I personally like the plastic notes. They re smoother and feel more durable than the paper notes. I ve had one go through a laundry load by accident and it came out the other side fine.
Another gentleman responded with slightly more information which probably would interest travellers from around the world, not just Indians
Quebec has its own interbank system called Interac (https://interac.ca/en/about/our-company.html). Quebec is a very proud and independent region and for many historical reasons they want to stand on their own, which is why they support their local systems. Some vendors will support only Interac for debit card transactions (at least this was the case when I stayed there the beginning of this decade, it might have changed a bit). *Most* vendors (including supermarkets like Provigo, Metro, etc) will accept major credit and debit cards, although MasterCard isn t accepted as widely there as Visa is. So, if you have one of both, load up your Visa card instead of your MasterCard or get a prepaid Visa card from your bank. They support chip cards everywhere so don t worry about that. If you have a 5 digit pin on any of your cards and a vendor asks you for a 4 digit pin, it will work 90%+ of the times if you just enter the first 4 digits, but it s usually a good idea to go change your pin to a 4 digit just to be safe.
From the Indian perspective all of the above fits pretty neat as we also have Pin and Chip cards (domestically though most ATMs still use the magnetic strip and is suspected that the POS terminals aren t any better.) That would be a whole different story so probably left for another day. I do like the bit about pocketing the change tip. As far as number of free transactions go, it was pretty limited in India for few years before the demonetization happening now. Few years before, I do remember doing as many transactions on the ATM as I please but then ATM s have seen a downward spiral in terms of technology upgradation, maintenance etc. There is no penalty to the bank if the ATM is out-of-order. If there was significant penalty then we probably would have seen banks taking more care of ATM s. It is a slightly more complex topic hence would take a stab at it some other day. Do hope though that the terms for ATM usage for bank customers become lenient similar to Canada otherwise it would be difficult for Indians to jump on the digital band-wagon as you cannot function without cheap, user-friendly technology. Cash machines: Uneven spread, slowing growth - Copyright Indian Express The image has been taken from this fascinating article which appeared in Indian Express couple of days back. Coming back to the cheese and wine in the evening. I think we started coming back from Eagle Encounters around 16:30/17:00 hrs Cape Town time. Somehow the ride back was much more faster and we played some Bollywood party music while coming back (all cool). Suddenly remembered that I had to buy some cheese as I hadn t bought any from India. There is quite a bit of a post where I m trying to know/understand if spices can be smuggled (which much later I learnt I didn t need to but that s a different story altogether), I also had off-list conversations with people about cheese as well but wasn t able to get any good recommendations. Then saw that KK bought Mysore Pak (apparently she took a chance not declaring it) which while not being exactly cheese fit right into things. In her own words a South Indian ghee sweet fondly nicknamed the blocks of cholesterol and reason #3 for bypass surgery . KK So with Leonard s help we stopped at a place where it looked like a chain of stores. Each store was selling something. Seeing that, I was immediately transported to Connaught Place, Delhi Connaught Place, Delhi The image comes from http://planetden.com/food/roundabout-world-connaught-place-delhi which attempts to explain Connaught Place. While the article is okish, it lacks soul and not written like a Delhite would write or anybody who has spent a chunk having spent holidays at CP. Another day, another story, sorry. What I found interesting about the stores while they were next to each other, I also eyed an alcohol shop as well as an Adult/Sex shop. I asked Leonard as to how far we were from UCT and he replied hardly 5 minutes by car and was shocked to see both alcohol and a sex shop. While an alcohol shop some distance away from a college is understandable, there are few and far around Colleges all over India, but adult shops are a rarity. Unfortunately, none of us have any photos of the place as till that time everybody s phone was dead or just going to be dead and nobody had thought to bring a portable power pack to juice our mobile devices. A part of me was curious to see what the sex shop would have and look from inside, but as was with younger people didn t think it was appropriate. All of us except Jaminy and someone else (besides Leonard) decided to stay back, while the rest of us went inside to explore the stores. It took me sometime to make my way to the cheese corner and had no idea which was good and which wasn t. So with no idea of brands therein, the only way to figure out was the pricing. So bought two, one a larger 500 gm cheap piece and a smaller slightly more expensive one just to make sure that the Debian cheese team would be happy. We did have a mini-adventure as for sometime Jaminy was missing, apparently she went goofing off or went to freshen up or something and we were unable to connect with her as all our phones were dead or dying. Eventually we came back to UCT, barely freshened up when it was decided by our group to go and give our share of goodies to the cheese and wine party. When I went up to the room to share the cheese, came to know they needed a volunteer for cutting veggies etc. Having spent years seeing Yan Can Cook and having practised quite a bit tried to do some fancy decoration and some julian cutting but as we got dull knives and not much time, just did some plain old cutting
The salads

The Salads, partly done by me.

I have to share I had a fascinating discussion about cooking in Pressure Cookers. I was under the assumption that everybody knows how to use Pressure Cookers as they are one of the simplest ways to cook food without letting go of all the nutrients. At least, I believe this to be predominant in the Asian subcontinent and even the chinese have similar vessels for cooking. I use what is called the first generation Pressure Cooker. I have been using a 1.5 l Prestige Pressure Cooker over half a decade, almost used daily without issues. http://www.amazon.in/Prestige-Nakshatra-Aluminium-Pressure-Cooker/dp/B00CB7U1OU
Prestige 1.5 L Pressure Cooker

1.5 Litre Pressure Cooker with gasket and everything.

There are also induction pressure cookers nowadays in the Indian market and this model https://www.amazon.in/Prestige-Deluxe-Induction-Aluminum-Pressure/dp/B01KZVPNGE/ref=sr_1_2
Induction base cooking for basmati rice

Best cooker for doing Basmati Biryanis and things like that.

Basmati is long-grain, aromatic rice which most families used in very special occasions such as festivals, marriages, anything good and pure is associated with the rice. I had also shared my lack of knowledge of industrial Microwave Ovens. While I do get most small Microwave Ovens like these , cooking in industrial ovens I simply have no clue. Anyways, after that conversation I went back, freshened up a bit and sometime later found myself in the middle of this
Collection of Wine Bottles

Random selection of wine bottles from all over the world.

Also at times found myself in middle of this
Chocolates all around me.

CHOCOLATES

I tried quite a few chocolates but the best one I liked (don t remember the name) was a white caramel chocolate which literally melted into my mouth. Got the whole died and went to heaven experience . Who said gluttony is bad Or this
French Bread, Wine and chaos

French Bread, Wine and chaos

As can be seen the French really enjoy their bread. I do remember a story vaguely (don t remember if it was a children s fairy tale or something) about how the French won a war through their french bread. Or this
Juices for those who love their health

Juices for those who love their health

We also had juices for the teetotaller or who can t handle drinks. Unsurprisingly perhaps, by the end of the session, almost all the different wines were finito while there was still some juices left to go around. From the Indian perspective, it wasn t at all exciting, there were no brawls, everybody was too civilized and everybody staggered off when they met their quota. As I was in holiday spirit, stayed up late, staggered to my room, blissed out and woke up without any headache. Pro tip Drink lots and lots and lots of water especially if you are drinking. It flushes out most of the toxins and also helps in not having after-morning headaches. If I m going drinking, I usually drown myself in at least a litre or two of water, even if I had to the bathroom couple of times before going to bed. All in all, a perfect evening. I was able to connect/talk with some of the gods whom I had wanted to for a long time and they actually listened. Don t remember if I mumbled something or made some sense in small-talk or whatever I did. But as shared, a perfect evening
Filed under: Miscellenous Tagged: #ATM usage, #Canada, #Cheese and Wine party, #Cheese shopping, #Connaught Place Delhi, #Debconf16, #Debit card, #French bread, #Julian cutting, #Mysore Pak, #white caramel chocolate

13 October 2016

Wouter Verhelst: Webserver certificate authentication with intermediate CAs

Authenticating HTTPS clients using TLS certificates has long been very easy to do. The hardest part, in fact, is to create a PKI infrastructure robust enough so that compromised (or otherwise no longer desirable) certificates cannot be used anymore. While setting that up can be fairly involved, it's all pretty standard practice these days. But authentication using a private CA is laughably easy. With apache, you just do something like:
SSLCACertificateFile /path/to/ca-certificate.pem
SSLVerifyClient require
...and you're done. With the above two lines, apache will send a CertificateRequest message to the client, prompting the client to search its local certificate store for certificates signed by the CA(s) in the ca-certificate.pem file, and using the certificate thus found to authenticate against the server. This works perfectly fine for setting up authentication for a handful of users. It will even work fine for a few thousands of users. But what if you're trying to set up website certificate authentication for millions of users? Well, in that case, storing everything in a single CA just doesn't work, and you'll need intermediate CAs to make things not fall flat on their face. Unfortunately, the standard does not state what should happen when a client certificate is signed by an intermediate certificate, and the distinguished name(s) in the CertificateRequest message are those of the certificate(s) at the top of the chain rather than the intermediates. Previously, browsers would not just send out the client certificate, but also send along the certificate that it knew to have signed that client certificate, and so on until it found a self-signed certificate. With that, the server would see a chain of certificates that it could verify against the root certificates in its local trust store, and certificate verification would succeed. It would appear that browsers are currently changing this behaviour, however. With the switch to BoringSSL, Google Chrome on the GNU/Linux platform no longer sent the signing certificates, and instead only sends the leaf certificate that it wants to use for certificate authentication. In the bug report, Ryan Sleevi explains that while the change was premature, the long term plans are for this to be the behaviour not just on GNU/Linux, but on Windows and macOS too. Meanwhile, the issue has been resolved for Chrome 54 (due to be released), but there's no saying for how long. As if that was not enough, the new version of Safari as shipped with macOS 10.12 has also stopped sending intermediate certificates, and expects the web server to be happy with just receiving the leaf certificates. So, I guess it's safe to say that when you want to authenticate a client in a multi-level hierarchical CA environment, you cannot just hand the webserver a list of root certificates and expect things to work anymore; you'll have to hand it the intermediary certificates as well. To do so, you need to modify the SSLCACertificateFile parameter in apache configuration so that it not only contains the root certificate, but all the intermediate certificates as well. If your list of intermediate certificates is rather long, it may improve performance to use SSLCACertificatePath and the c_rehash tool to create a hashed directory with certificates rather than a PEM file, but in essense, that should work. While this works, the problem with doing so is that now the DNs of all the intermediate certificates are sent along with the CertificateRequest message to the client, which (again, if your list of intermediate certificates is rather long) may result in performance issues. The fix for that is fairly easy: add a line like
SSLCADNRequestFile /path/to/root-certificates.pem
where the file root-certificates.pem contains the root certificates only. This file will tell the webserver which certificates should be announced in the CertificateRequest message, but the SSLCACertificateFile (or SSLCACertificatePath) configuration item will still be used for actual verification of the certificates in question. Note though that the root certificates apparently also seem to need to be available in the SSLCACertificatePath or SSLCACertificateFile configuration; if you do not do so, then authentication seems to fail, although I haven't yet found why. I've set up a test page for all of those "millions" of certificates, and it seems to work for me. I've you're trying to use one of those millions of certificates against your own webserver or have a similar situation with a different set of certificates, you might want to make the above changes, too.

9 October 2016

Ben Armstrong: Annual Hike with Ryan: Salt Marsh Trail, 2016

Once again, Ryan Neily and I met last month for our annual hike. This year, to give our aging knees a break, we visited the Salt Marsh Trail for the first time. For an added level of challenge and to access the trail by public transit, we started with the Shearwater Flyer Trail and finished with the Heritage Trail. It was a perfect day both for hiking and photography: cool with cloud cover and a refreshing coastal breeze. The entire hike was over 25 km and took the better part of the day to complete. Good times, great conversations, and I look forward to visiting these beautiful trails again!
Salt Marsh trail hike, 2016. Click to start the slideshow.Salt Marsh trail hike, 2016. Click to start the slideshow.
We start here, on the Shearwater flyer trail.We start here, on the Shearwater flyer trail.
Couldn t ID this bush. The berries are spectacular! A pond to the side of the trail. Different angle for dramatic lighting effect. Rail bridge converted to foot bridge. Cranberries! Reviewing our progress. From the start Map of the Salt Marsh trail ahead. Off we go again! First glimpse through the trees. Appreciating the cloud cover today. Salt-marshy grasses. Never far from rocks in NS. Rocks all laid out in stripes. Lunch & selfie time. Ryan attacking his salad. Vantage point. A bit of causeway coast. Plenty of eel grass. Costal flora. We head for the bridge next. Impressed by the power of the flow beneath. Snapping more marsh shots. Ripples. Gulls, and if you squint, a copter. More ripples. Swift current along this channel. Until it broadens out and slows down. Nearly across. Heron! Sorry it s so tiny. Heron again, before I lost it. Ducks at the head of the Atlantic View trail where we rested and then turned back. Attempt at artsy. Nodding ladies tresses on the way back. Several of them. Sky darkening, but we still have time. A lonely wild rose. The last gasp of late summer. Back across the marshes. A short breather on the Heritage Trail.
Here s the Strava record of our hike:
Facebooktwittergoogle_plusredditpinterestlinkedinmail

7 July 2016

Matthew Garrett: Bluetooth LED bulbs

The best known smart bulb setups (such as the Philips Hue and the Belkin Wemo) are based on Zigbee, a low-energy, low-bandwidth protocol that operates on various unlicensed radio bands. The problem with Zigbee is that basically no home routers or mobile devices have a Zigbee radio, so to communicate with them you need an additional device (usually called a hub or bridge) that can speak Zigbee and also hook up to your existing home network. Requests are sent to the hub (either directly if you're on the same network, or via some external control server if you're on a different network) and it sends appropriate Zigbee commands to the bulbs.

But requiring an additional device adds some expense. People have attempted to solve this in a couple of ways. The first is building direct network connectivity into the bulbs, in the form of adding an 802.11 controller. Go through some sort of setup process[1], the bulb joins your network and you can communicate with it happily. Unfortunately adding wifi costs more than adding Zigbee, both in terms of money and power - wifi bulbs consume noticeably more power when "off" than Zigbee ones.

There's a middle ground. There's a large number of bulbs available from Amazon advertising themselves as Bluetooth, which is true but slightly misleading. They're actually implementing Bluetooth Low Energy, which is part of the Bluetooth 4.0 spec. Implementing this requires both OS and hardware support, so older systems are unable to communicate. Android 4.3 devices tend to have all the necessary features, and modern desktop Linux is also fine as long as you have a Bluetooth 4.0 controller.

Bluetooth is intended as a low power communications protocol. Bluetooth Low Energy (or BLE) is even lower than that, running in a similar power range to Zigbee. Most semi-modern phones can speak it, so it seems like a pretty good choice. Obviously you lose the ability to access the device remotely, but given the track record on this sort of thing that's arguably a benefit. There's a couple of other downsides - the range is worse than Zigbee (but probably still acceptable for any reasonably sized house or apartment), and only one device can be connected to a given BLE server at any one time. That means that if you have the control app open while you're near a bulb, nobody else can control that bulb until you disconnect.

The quality of the bulbs varies a great deal. Some of them are pure RGB bulbs and incapable of producing a convincing white at a reasonable intensity[2]. Some have additional white LEDs but don't support running them at the same time as the colour LEDs, so you have the choice between colour or a fixed (and usually more intense) white. Some allow running the white LEDs at the same time as the RGB ones, which means you can vary the colour temperature of the "white" output.

But while the quality of the bulbs varies, the quality of the apps doesn't really. They're typically all dreadful, competing on features like changing bulb colour in time to music rather than on providing a pleasant user experience. And the whole "Only one person can control the lights at a time" thing doesn't really work so well if you actually live with anyone else. I was dissatisfied.

I'd met Mike Ryan at Kiwicon a couple of years back after watching him demonstrate hacking a BLE skateboard. He offered a couple of good hints for reverse engineering these devices, the first being that Android already does almost everything you need. Hidden in the developer settings is an option marked "Enable Bluetooth HCI snoop log". Turn that on and all Bluetooth traffic (including BLE) is dumped into /sdcard/btsnoop_hci.log. Turn that on, start the app, make some changes, retrieve the file and check it out using Wireshark. Easy.

Conveniently, BLE is very straightforward when it comes to network protocol. The only thing you have is GATT, the Generic Attribute Protocol. Using this you can read and write multiple characteristics. Each packet is limited to a maximum of 20 bytes. Most implementations use a single characteristic for light control, so it's then just a matter of staring at the dumped packets until something jumps out at you. A pretty typical implementation is something like:

0x56,r,g,b,0x00,0xf0,0x00,0xaa

where r, g and b are each just a single byte representing the corresponding red, green or blue intensity. 0x56 presumably indicates a "Set the light to these values" command, 0xaa indicates end of command and 0xf0 indicates that it's a request to set the colour LEDs. Sending 0x0f instead results in the previous byte (0x00 in this example) being interpreted as the intensity of the white LEDs. Unfortunately the bulb I tested that speaks this protocol didn't allow you to drive the white LEDs at the same time as anything else - setting the selection byte to 0xff didn't result in both sets of intensities being interpreted at once. Boo.

You can test this out fairly easily using the gatttool app. Run hcitool lescan to look for the device (remember that it won't show up if anything else is connected to it at the time), then do gatttool -b deviceid -I to get an interactive shell. Type connect to initiate a connection, and once connected send commands by doing char-write-cmd handle value using the handle obtained from your hci dump.

I did this successfully for various bulbs, but annoyingly hit a problem with one from Tikteck. The leading byte of each packet was clearly a counter, but the rest of the packet appeared to be garbage. For reasons best known to themselves, they've implemented application-level encryption on top of BLE. This was a shame, because they were easily the best of the bulbs I'd used - the white LEDs work in conjunction with the colour ones once you're sufficiently close to white, giving you good intensity and letting you modify the colour temperature. That gave me incentive, but figuring out the protocol took quite some time. Earlier this week, I finally cracked it. I've put a Python implementation on Github. The idea is to tie it into Ulfire running on a central machine with a Bluetooth controller, making it possible for me to control the lights from multiple different apps simultaneously and also integrating with my Echo.

I'd write something about the encryption, but I honestly don't know. Large parts of this make no sense to me whatsoever. I haven't even had any gin in the past two weeks. If anybody can explain how anything that's being done there makes any sense at all[3] that would be appreciated.

[1] typically via the bulb pretending to be an access point, but also these days through a terrifying hack involving spewing UDP multicast packets of varying lengths in order to broadcast the password to associated but unauthenticated devices and good god the future is terrifying

[2] For a given power input, blue LEDs produce more light than other colours. To get white with RGB LEDs you either need to have more red and green LEDs than blue ones (which costs more), or you need to reduce the intensity of the blue ones (which means your headline intensity is lower). Neither is appealing, so most of these bulbs will just give you a blue "white" if you ask for full red, green and blue

[3] Especially the bit where we calculate something from the username and password and then encrypt that using some random numbers as the key, then send 50% of the random numbers and 50% of the encrypted output to the device, because I can't even

comment count unavailable comments

27 April 2016

Niels Thykier: auto-decrufter in top 5 after 10 months

About 10 months ago, we enabled an auto-decrufter in dak. Then after 3 months it had become the top 11th remover . Today, there are only 3 humans left that have removed more packages than the auto-decrufter impressively enough, one of them is not even an active FTP-master (anymore). The current score board:
 5371 Luca Falavigna
 5121 Alexander Reichle-Schmehl
 4401 Ansgar Burchardt
 3928 DAK's auto-decrufter
 3257 Scott Kitterman
 2225 Joerg Jaspert
 1983 James Troup
 1793 Torsten Werner
 1025 Jeroen van Wolffelaar
  763 Ryan Murray
For comparison, here is the number removals by year for the past 6 years:
 5103 2011
 2765 2012
 3342 2013
 3394 2014
 3766 2015  (1842 removed by auto-decrufter)
 2845 2016  (2086 removed by auto-decrufter)
Which tells us that in 2015, the FTP masters and the decrufter performed on average over 10 removals a day. And by the looks of it, 2016 will surpass that. Of course, the auto-decrufter has a tendency to increase the number of removed items since it is an advocate of remove early, remove often! .:) Data is from https://ftp-master.debian.org/removals-full.txt. Scoreboard computed as:
  grep ftpmaster: removals-full.txt   \
   perl -pe 's/.*ftpmaster:\s+//; s/\]$//;'   \
   sort   uniq -c   sort --numeric --reverse   head -n10
Removals by year computed as:
 grep ftpmaster: removals-full.txt   \
   perl -pe 's/.* (\d 4 ) \d 2 :\d 2 :\d 2 .*/$1/'   uniq -c   tail -n6
(yes, both could be done with fewer commands)
Filed under: Debian

16 April 2016

Ryan Kavanagh: Specifying a custom MTA path in caff

I recently had to sign someone s GPG key. I ve long used the caff tool from the signing-party package to help me with this. Unfortunately, I m using a new laptop and hadn t yet configured caff on it. Moreover, caff uses the system MTA by default, normally found at /usr/sbin/sendmail, and I hadn t yet properly configured it to send mail to the outside world. Since I have multiple email accounts and use mutt as my mail client, I use msmtp as my SMTP client / sendmail drop-in. This post describes how to configure caff to use msmtp. Reading the caff man page, one sees the following tantalising hint, which leads you to believe it s completely trivial to specify your own MTA:
$CONFIG 'mailer-send'  = [ 'sendmail', '-f', $CONFIG 'email' , '-it' ];
Unfortunately, substituting in a custom path, e.g., /home/rak/bin/msmtp/msmtp-default, for sendmail in the above line results in a long sequence of errors when caff tries to mail the signed keys:
Bareword "home" not allowed while "strict subs" in use at (eval 218) line 1.
Bareword "rak" not allowed while "strict subs" in use at (eval 218) line 1.
Bareword "bin" not allowed while "strict subs" in use at (eval 218) line 1.
Bareword "msmtp" not allowed while "strict subs" in use at (eval 218) line 1.
Bareword "msmtp" not allowed while "strict subs" in use at (eval 218) line 1.
Bareword "default" not allowed while "strict subs" in use at (eval 218) line 1.
Fortunately, it s still straightforward to accomplish after reading the perldoc for Mail::Mailer. In my case, it was sufficient to add the following to my ~/.caffrc:
$ENV 'PERL_MAILERS'  = 'sendmail:'.$ENV 'HOME' .'/bin/msmtp/msmtp-default';
$CONFIG 'mailer-send'  = [ 'sendmail' ];
To specify alternate paths to your sendmail-style MTA, simply modify the value of $ENV 'PERL_MAILERS' after the initial sendmail: bit. If you re using stock msmtp, I believe the following pair of lines will work, though I haven t tested it:
$ENV 'PERL_MAILERS'  = 'sendmail:/usr/bin/msmtp';
$CONFIG 'mailer-send'  = [ 'sendmail' ];
If there s a more elegant way to accomplish this, I d be happy to hear about it!

1 April 2016

Jonathan Dowland: April Fools

I'm not a big fan of April Fools' day, but this year I decided to try one. My modest contribution: a Pull Request for Chocolate Doom titled Bring back the classic Doom "hit enter" behaviour:
When playing PWADs with classic Doom, you always got the "this game has been modified" banner and had to hit enter to continue. If you played a lot of PWADs, you got used to hitting enter regularly. It became a core part of the Doom-playing experience. Even if you used one of those lovely launchers that came on the shovelware CDs which streamlined the PWAD choosing experience, you always had to hit the enter key. This commit restores this classic behaviour and allows those who still have the muscle-memory "hit enter" behaviour permanently burned into their nervous systems to finally be at peace. TODO for a future commit: the coloured top-line of text (red for 1.666, blue for 1.9 iirc)
Some feedback from IRC:
<Linguica> its funny because its nearly indistinguishable from real chocolate doom feature requests <jmickle> i actually first thought "huh, kinda surprised this isn't in already <ryan> best chocolate-doom feature yet! ;)

21 March 2016

Lunar: Reproducible builds: week 47 in Stretch cycle

What happened in the reproducible builds effort between March 13th and March 19th 2016:

Toolchain fixes
  • Petter Reinholdtsen uploaded naturaldocs/1.51-1.1 which makes the output reproducible. Original patch by Chris Lamb.
  • Damyan Ivanov uploaded libpdf-api2-perl/2.025-2 which will make internal font ID reproducible.
  • Christian Hofstaedtler uploaded ruby2.3/2.3.0-5 which sets gzip embedded mtime field to fixed value for rdoc-generated compressed javascript data.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: diction, doublecmd, ruby-hiredis, vdr-plugin-epgsearch. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #818128 on nethack by Reiner Herrmann: implement support for SOURCE_DATE_EPOCH, set LC_ALL to C, and ensure deterministic build order when running parallel builds.
  • #818111 on debian-keyring by Satyam Zode: fix the order of files in md5sums.
  • #818067 on ncurses by Niels Thykier: strip trailing whitespaces introduced when using dash as system shell.
  • #818230 on aircrack-ng by Reiner Herrmann: build assembly code as a separate .o file.
  • #818419 on mutt by Daniel Shahaf: use C locale when listing files to be put in README.Patches.
  • #818430 on ruby-coveralls by Dhole: ensure UTC is used as the timezone when generating the documentation.
  • #818686 on littlewizard by Reiner Herrmann: use the C locale in the script for iterating over the files.
  • #818704 on strigi by Reiner Herrmann: sort keys when traversing hashes in makecode.pl.

Package reviews 44 reviews have been removed, 40 added and 5 updated in the previous week. Chris Lamb has reported 16 FTBFS.

1 February 2016

Lunar: Reproducible builds: week 40 in Stretch cycle

What happened in the reproducible builds effort between January 24th and January 30th:

Media coverage Holger Levsen was interviewed by the FOSDEM team to introduce his talk on Sunday 31st.

Toolchain fixes Jonas Smedegaard uploaded d-shlibs/0.63 which makes the order of dependencies generated by d-devlibdeps stable accross locales. Original patch by Reiner Herrmann.

Packages fixed The following 53 packages have become reproducible due to changes in their build dependencies: appstream-glib, aptitude, arbtt, btrfs-tools, cinnamon-settings-daemon, cppcheck, debian-security-support, easytag, gitit, gnash, gnome-control-center, gnome-keyring, gnome-shell, gnome-software, graphite2, gtk+2.0, gupnp, gvfs, gyp, hgview, htmlcxx, i3status, imms, irker, jmapviewer, katarakt, kmod, lastpass-cli, libaccounts-glib, libam7xxx, libldm, libopenobex, libsecret, linthesia, mate-session-manager, mpris-remote, network-manager, paprefs, php-opencloud, pisa, pyacidobasic, python-pymzml, python-pyscss, qtquick1-opensource-src, rdkit, ruby-rails-html-sanitizer, shellex, slony1-2, spacezero, spamprobe, sugar-toolkit-gtk3, tachyon, tgt. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them:
  • gnubg/1.05.000-4 by Russ Allbery.
  • grcompiler/4.2-6 by Hideki Yamane.
  • sdlgfx/2.0.25-5 fix by Felix Geyer, uploaded by Gianfranco Costamagna.
Patches submitted which have not made their way to the archive yet:
  • #812876 on glib2.0 by Lunar: ensure that functions are sorted using the C locale when giotypefuncs.c is generated.

diffoscope development diffoscope 48 was released on January 26th. It fixes several issues introduced by the retrieval of extra symbols from Debian debug packages. It also restores compatibility with older versions of binutils which does not support readelf --decompress.

strip-nondeterminism development strip-nondeterminism 0.015-1 was uploaded on January 27th. It fixes handling of signed JAR files which are now going to be ignored to keep the signatures intact.

Package reviews 54 reviews have been removed, 36 added and 17 updated in the previous week. 30 new FTBFS bugs have been submitted by Chris Lamb, Michael Tautschnig, Mattia Rizzolo, Tobias Frost.

Misc. Alexander Couzens and Bryan Newbold have been busy fixing more issues in OpenWrt. Version 1.6.3 of FreeBSD's package manager pkg(8) now supports SOURCE_DATE_EPOCH. Ross Karchner did a lightning talk about reproducible builds at his work place and shared the slides.

11 November 2015

Jonathan Dowland: Useful Mac programs

A little over a year ago I wrote about how I'd been using a Mac as my main work machine. I hadn't written anything on the subject since. Here are four useful Mac programs that I can recommend to people. Sizeup, from Irradiated Software. A bit Like Windows' aero snap, but on steroids. I love this. I regularly move windows between two desktops (external and internal display), resize and centre them or put them in one quarter of the screen with just a few presses. The "snapback" feature is also great. X Lossless Decoder (XLD). A handy transcoder that can use QuickTime encoders and so can write out Apple/Quicktime/iTunes-encoded AAC/MP4 files en-mass, translating file meta-data.
LimeChat LimeChat
LimeChat, an IRC client. I theme it to look like Colloquy. The net result is pretty clean. I like using a proportional-font for text things, so getting IRC out of a terminal is a win for me. LimeChat automatically fetches and thumbnails URIs to images that people paste in channels, which is either incredibly convenient or a curse, depending on the channel. You can toggle that behaviour but only across the whole client, not on a channel or network basis. Finally, 1Password is an incredibly slick password manager. I use it in a very basic fashion: no mobile client, no syncing to anything outside of my workstation. You could also use LastPass which is similar and has a Linux client. I haven't tried it, but there's a third party tool to read 1Password password stores on Linux written by Ryan Coleman of SDL fame called 1pass.

27 October 2015

Lunar: Reproducible builds: week 26 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Mattia Rizzolo created a bug report to continue the discussion on storing cryptographic checksums of the installed .deb in dpkg database. This follows the discussion that happened in June and is a pre-requisite to add checksums to .buildinfo files. Niko Tyni identified why the Vala compiler would generate code in varying order. A better patch than his initial attempt still needs to be written. Packages fixed The following 15 packages became reproducible due to changes in their build dependencies: alt-ergo, approx, bin-prot, caml2html, coinst, dokujclient, libapreq2, mwparserfromhell, ocsigenserver, python-cryptography, python-watchdog, slurm-llnl, tyxml, unison2.40.102, yojson. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: reproducible.debian.net pbuilder has been updated to version 0.219~bpo8+1 on all eight build nodes. (Mattia Rizzolo, h01ger) Packages that FTBFS but for which no open bugs have been recorded are now tested again after 3 days. Likewise for depwait packages. (h01ger) Out of disk situations will not cause IRC notifications anymore. (h01ger) Documentation update Lunar continued to work on writing documentation for the future reproducible-builds.org website. Package reviews 44 reviews have been removed, 81 added and 48 updated this week. Chris West and Chris Lamb identified 70 fail to build from source issues. Misc. h01ger presented the project in Mexico City at the 3er Congreso de Seguridad de la Informaci n where it became clear that we lack academic papers related to reproducible builds. Bryan has been doing hard work to improve reproducibility for OpenWrt. He wrote a report linking to the patches and test results he published.

16 October 2015

Dirk Eddelbuettel: R / Finance 2016 Call for Papers

Earlier today, Josh sent the text below in this message to the R-SIG-Finance list as the very first heads-up concerning the 2016 edition of our successful R/Finance series. We are once again very excited about our conference, thrilled about upcoming keynotes (some of which are confirmed and some of which are in the works), and hope that many R / Finance users will not only join us in Chicago in May 2016 -- but also submit an exciting proposal. So read on below, and see you in Chicago in May!
Call for Papers R/Finance 2016: Applied Finance with R
May 20 and 21, 2016
University of Illinois at Chicago, IL, USA The eight annual R/Finance conference for applied finance using R will be held on May 20 and 21, 2016, in Chicago, IL, USA at the University of Illinois at Chicago. The conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure, and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management, portfolio construction, and trading. Over the past seven years, R/Finance has included attendees from around the world. It has featured presentations from prominent academics and practitioners, and we anticipate another exciting line-up for 2016. We invite you to submit complete papers in pdf format for consideration. We will also consider one-page abstracts (in txt or pdf format) although more complete papers are preferred. We welcome submissions for both full talks and abbreviated "lightning talks." Both academic and practitioner proposals related to R are encouraged. All slides will be made publicly available at conference time. Presenters are strongly encouraged to provide working R code to accompany the slides. Data sets should also be made public for the purposes of reproducibility (though we realize this may be limited due to contracts with data vendors). Preference may be given to presenters who have released R packages. The conference will award two (or more) $1000 prizes for best papers. A submission must be a full paper to be eligible for a best paper award. Extended abstracts, even if a full paper is provided by conference time, are not eligible for a best paper award. Financial assistance for travel and accommodation may be available to presenters, however requests must be made at the time of submission. Assistance will be granted at the discretion of the conference committee. Please make your submission online at this link. The submission deadline is January 29, 2016. Submitters will be notified via email by February 29, 2016 of acceptance, presentation length, and financial assistance (if requested). Additional details will be announced via the R/Finance conference website as they become available. Information on previous years' presenters and their presentations are also at the conference website. For the program committee:
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich

27 September 2015

Ben Armstrong: Annual Bluff Hike, 2015

Here is a photo journal of our hike on the Bluff Wilderness Trail with my friend, Ryan Neily, as is our tradition at this time of year. Rather than hike all four loops, as we achieved last year, we chose to cover only the Pot Lake and Indian Hill loops. Like our meandering pace, our conversations were enjoyable and far ranging, with Nature doing her part, stimulating our minds and bodies and refreshing our spirits.
A break at the summit of Pot Lake loop. Click to start slideshow.A break at the summit of Pot Lake loop. Click to start slideshow.
Northern bayberry A few showers quickly dissipated into light mist on the first leg of the hike Ryan, enjoying one of the many beautiful views Cormorant or shag. Hard to say from this poor, zoomed cellphone shot. Darkened pool amongst the rugged trees Late summer colours A riot of life shoots up in every crevice Large boulders and trees, forming a non-concrete alley along the trail margin Huckleberries still plentiful on the Indian Hill loop Sustenance to keep us going Not at all picked over, like the Pot Lake loop We break here for lunch Just about ready to embark on the last half We are surprised by the productivity of these short, scrubby huckleberries Barely rising from the reindeer moss, each huckleberry twig provides sweet, juicy handfuls A small pond on the trip back A break on the home stretch Common juniper, which nevertheless is not so common out here Immature green common juniper berries (actually cones)
facebooktwittergoogle_plusredditpinterestlinkedinmail

24 September 2015

Joachim Breitner: The Incredible Proof Machine

In a few weeks, I will have the opportunity to offer a weekend workshop to selected and motivated high school students1 to a topic of my choice. My idea is to tell them something about logic, proofs, and the joy of searching and finding proofs, and the gratification of irrevocable truths. While proving things on paper is already quite nice, it is much more fun to use an interactive theorem prover, such as Isabelle, Coq or Agda: You get immediate feedback, you can experiment and play around if you are stuck, and you get lots of small successes. Someone2 once called interactive theorem proving the worlds most geekiest videogame . Unfortunately, I don t think one can get high school students without any prior knowledge in logic, or programming, or fancy mathematical symbols, to do something meaningful with a system like Isabelle, so I need something that is (much) easier to use. I always had this idea in the back of my head that proving is not so much about writing text (as in normally written proofs) or programs (as in Agda) or labeled statements (as in Hilbert-style proofs), but rather something involving facts that I have proven so far floating around freely, and way to combine these facts to new facts, without the need to name them, or put them in a particular order or sequence. In a way, I m looking for labVIEW wrestled through the Curry-Horward-isomorphism. Something like this:
A proof of implication currying

A proof of implication currying

So I set out, rounded up a few contributors (Thanks!), implemented this, and now I proudly present: The Incredible Proof Machine3 This interactive theorem prover allows you to do perform proofs purely by dragging blocks (representing proof steps) onto the paper and connecting them properly. There is no need to learn syntax, and hence no frustration about getting that wrong. Furthermore, it comes with a number of example tasks to experiment with, so you can simply see it as a challenging computer came and work through them one by one, learning something about the logical connectives and how they work as you go. For the actual workshop, my plan is to let the students first try to solve the tasks of one session on their own, let them draw their own conclusions and come up with an idea of what they just did, and then deliver an explanation of the logical meaning of what they did. The implementation is heavily influenced by Isabelle: The software does not know anything about, say, conjunction ( ) and implication ( ). To the core, everything is but an untyped lambda expression, and when two blocks are connected, it does unification4 of the proposition present on either side. This general framework is then instantiated by specifying the basic rules (or axioms) in a descriptive manner. It is quite feasible to implement other logics or formal systems on top of this as well. Another influence of Isabelle is the non-linear editing: You neither have to create the proof in a particular order nor have to manually manage a proof focus . Instead, you can edit any bit of the proof at any time, and the system checks all of it continuously. As always, I am keen on feedback. Also, if you want to use this for your own teaching or experimenting needs, let me know. We have a mailing list for the project, the code is on GitHub, where you can also file bug reports and feature requests. Contributions are welcome! All aspects of the logic are implemented in Haskell and compiled to JavaScript using GHCJS, the UI is plain hand-written and messy JavaScript code, using JointJS to handle the graph interaction. Obviously, there is still plenty that can be done to improve the machine. In particular, the ability to create your own proof blocks, such as proof by contradiction, prove them to be valid and then use them in further proofs, is currently being worked on. And while the page will store your current progress, including all proofs you create, in your browser, it needs better ways to save, load and share tasks, blocks and proofs. Also, we d like to add some gamification, i.e. achievements ( First proof by contradiction , 50 theorems proven ), statistics, maybe a share theorem on twitter button. As the UI becomes more complicated, I d like to investigating moving more of it into Haskell world and use Functional Reactive Programming, i.e. Ryan Trickle s reflex, to stay sane. Customers who liked The Incredible Proof Machine might also like these artifacts, that I found while looking whether something like this exists:

  1. Students with migration background supported by the START scholarship
  2. Does anyone know the reference?
  3. We almost named it Proofcraft , which would be a name our current Minecraft-wild youth would appreciate, but it is alreay taken by Gerwin Kleins blog. Also, the irony of a theorem prover being in-credible is worth something.
  4. Luckily, two decades ago, Tobias Nipkow published a nice implementation of higher order pattern unification as ML code, which I transliterated to Haskell for this project.

Next.

Previous.